本文在3D Point Cloud中介绍了一个新问题：很少示例实例分割。给定一些带注释的点云举例说明了目标类，我们的目标是在查询点云中细分该目标类的所有实例。这个问题具有广泛的实用应用，在重点实例分段注释非常昂贵的收集中。为了解决此问题，我们提出了测量形式 - 第一个用于3D点云实例分割的地球引导变压器。关键的想法是利用大地距离来应对LIDAR 3D点云的密度不平衡。 LIDAR 3D点云在物体表面附近茂密，在其他地方稀疏或空，使欧几里得距离较差以区分不同的物体。另一方面，大地测量距离更合适，因为它编码了场景的几何形状，该几何形状可以用作变压器解码器中注意机制的指导信号，以生成代表实例的不同特征的内核。然后将这些内核用于动态卷积以获得最终实例掩模。为了评估新任务上的测量形式，我们提出了两个常见的3D点云实例分割数据集的新拆分：ScannETV2和S3DIS。地球形式始终优于根据最新的3D点云实例分割方法的强大基线，并具有明显的余量。代码可从https://github.com/vinairesearch/geoformer获得。

在本文中，我们采用了最大化的互信息（MI）方法来解决无监督的二进制哈希代码的问题，以实现高效的跨模型检索。我们提出了一种新颖的方法，被称为跨模型信息最大散列（CMIMH）。首先，要学习可以保留模跨和跨间相似性的信息的信息，我们利用最近估计MI的变分的进步，以最大化二进制表示和输入特征之间的MI以及不同方式的二进制表示之间的MI。通过在假设由多变量Bernoulli分布模型的假设下联合最大化这些MIM，我们可以学习二进制表示，该二进制表示，其可以在梯度下降中有效地以微量批量方式有效地保留帧内和模态的相似性。此外，我们发现尝试通过学习与来自不同模式的相同实例的类似二进制表示来最小化模态差距，这可能导致更少的信息性表示。因此，在减少模态间隙和失去模态 - 私人信息之间平衡对跨模型检索任务很重要。标准基准数据集上的定量评估表明，该方法始终如一地优于其他最先进的跨模型检索方法。

脱机策略学习（OPL）利用现有数据收集了策略优化的先验，而无需任何活动探索。尽管存在普遍性和近期对此问题的兴趣，但其函数近似设置中的理论和算法基础仍然持续开发。在本文中，我们考虑了在具有神经网络的离线上下文匪徒中的分布换档，优化和泛化轴上的这个问题。特别是，我们提出了一种可从线有效的离线情境匪徒，具有神经网络函数近似，不需要对奖励的任何功能假设。我们表明，我们的方法在较温和的情况下通过不良语境提供了比现有的OPL工作的分支变换。值得注意的是，与任何其他OPL方法不同，我们的方法使用随机梯度血统以在线方式从脱机数据中学习，允许我们利用在线学习的优势进入离线设置。此外，我们表明我们的方法更加计算效率，并且更好地依赖于神经网络的有效维度而不是在线对应物。最后，我们展示了我们在一系列合成和现实世界OPL问题中的方法的实证效果。

头部姿势估计是一个具有挑战性的任务，旨在解决与预测三维向量相关的问题，这为人机互动或客户行为中的许多应用程序提供服务。以前的研究提出了一些用于收集头部姿势数据的精确方法。但这些方法需要昂贵的设备，如深度摄像机或复杂的实验室环境设置。在这项研究中，我们引入了一种新的方法，以有效的成本和易于设置，以收集头部姿势图像，即UET-HEADBETS数据集，具有顶视图头姿势数据。该方法使用绝对方向传感器而不是深度摄像机快速设置，但仍然可以确保良好的效果。通过实验，我们的数据集已显示其分发和可用数据集之间的差异，如CMU Panoptic DataSet \ Cite {CMU}。除了使用UET符号数据集和其他头部姿势数据集外，我们还介绍了称为FSANET的全范围模型，这显着优于UET-HEALPETS数据集的头部姿势估计结果，尤其是在顶视图上。此外，该模型非常重量轻，占用小尺寸图像。

Diabetic Retinopathy (DR) is a leading cause of vision loss in the world, and early DR detection is necessary to prevent vision loss and support an appropriate treatment. In this work, we leverage interactive machine learning and introduce a joint learning framework, termed DRG-Net, to effectively learn both disease grading and multi-lesion segmentation. Our DRG-Net consists of two modules: (i) DRG-AI-System to classify DR Grading, localize lesion areas, and provide visual explanations; (ii) DRG-Expert-Interaction to receive feedback from user-expert and improve the DRG-AI-System. To deal with sparse data, we utilize transfer learning mechanisms to extract invariant feature representations by using Wasserstein distance and adversarial learning-based entropy minimization. Besides, we propose a novel attention strategy at both low- and high-level features to automatically select the most significant lesion information and provide explainable properties. In terms of human interaction, we further develop DRG-Net as a tool that enables expert users to correct the system's predictions, which may then be used to update the system as a whole. Moreover, thanks to the attention mechanism and loss functions constraint between lesion features and classification features, our approach can be robust given a certain level of noise in the feedback of users. We have benchmarked DRG-Net on the two largest DR datasets, i.e., IDRID and FGADR, and compared it to various state-of-the-art deep learning networks. In addition to outperforming other SOTA approaches, DRG-Net is effectively updated using user feedback, even in a weakly-supervised manner.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Non-invasive prostate cancer detection from MRI has the potential to revolutionize patient care by providing early detection of clinically-significant disease (ISUP grade group >= 2), but has thus far shown limited positive predictive value. To address this, we present an MRI-based deep learning method for predicting clinically significant prostate cancer applicable to a patient population with subsequent ground truth biopsy results ranging from benign pathology to ISUP grade group~5. Specifically, we demonstrate that mixed supervision via diverse histopathological ground truth improves classification performance despite the cost of reduced concordance with image-based segmentation. That is, where prior approaches have utilized pathology results as ground truth derived from targeted biopsies and whole-mount prostatectomy to strongly supervise the localization of clinically significant cancer, our approach also utilizes weak supervision signals extracted from nontargeted systematic biopsies with regional localization to improve overall performance. Our key innovation is performing regression by distribution rather than simply by value, enabling use of additional pathology findings traditionally ignored by deep learning strategies. We evaluated our model on a dataset of 973 (testing n=160) multi-parametric prostate MRI exams collected at UCSF from 2015-2018 followed by MRI/ultrasound fusion (targeted) biopsy and systematic (nontargeted) biopsy of the prostate gland, demonstrating that deep networks trained with mixed supervision of histopathology can significantly exceed the performance of the Prostate Imaging-Reporting and Data System (PI-RADS) clinical standard for prostate MRI interpretation.

Manually analyzing spermatozoa is a tremendous task for biologists due to the many fast-moving spermatozoa, causing inconsistencies in the quality of the assessments. Therefore, computer-assisted sperm analysis (CASA) has become a popular solution. Despite this, more data is needed to train supervised machine learning approaches in order to improve accuracy and reliability. In this regard, we provide a dataset called VISEM-Tracking with 20 video recordings of 30s of spermatozoa with manually annotated bounding-box coordinates and a set of sperm characteristics analyzed by experts in the domain. VISEM-Tracking is an extension of the previously published VISEM dataset. In addition to the annotated data, we provide unlabeled video clips for easy-to-use access and analysis of the data. As part of this paper, we present baseline sperm detection performances using the YOLOv5 deep learning model trained on the VISEM-Tracking dataset. As a result, the dataset can be used to train complex deep-learning models to analyze spermatozoa. The dataset is publicly available at https://zenodo.org/record/7293726.

The introduction of high-quality image generation models, particularly the StyleGAN family, provides a powerful tool to synthesize and manipulate images. However, existing models are built upon high-quality (HQ) data as desired outputs, making them unfit for in-the-wild low-quality (LQ) images, which are common inputs for manipulation. In this work, we bridge this gap by proposing a novel GAN structure that allows for generating images with controllable quality. The network can synthesize various image degradation and restore the sharp image via a quality control code. Our proposed QC-StyleGAN can directly edit LQ images without altering their quality by applying GAN inversion and manipulation techniques. It also provides for free an image restoration solution that can handle various degradations, including noise, blur, compression artifacts, and their mixtures. Finally, we demonstrate numerous other applications such as image degradation synthesis, transfer, and interpolation.

Artificial intelligence methods including deep neural networks (DNN) can provide rapid molecular classification of tumors from routine histology with accuracy that matches or exceeds human pathologists. Discerning how neural networks make their predictions remains a significant challenge, but explainability tools help provide insights into what models have learned when corresponding histologic features are poorly defined. Here, we present a method for improving explainability of DNN models using synthetic histology generated by a conditional generative adversarial network (cGAN). We show that cGANs generate high-quality synthetic histology images that can be leveraged for explaining DNN models trained to classify molecularly-subtyped tumors, exposing histologic features associated with molecular state. Fine-tuning synthetic histology through class and layer blending illustrates nuanced morphologic differences between tumor subtypes. Finally, we demonstrate the use of synthetic histology for augmenting pathologist-in-training education, showing that these intuitive visualizations can reinforce and improve understanding of histologic manifestations of tumor biology.